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ABSTRACT 

Over the past 30 years, researchers have repeatedly demonstrated the need to improve academic 
advising. Nonetheless, at many Universities academic advising remains a neglected endeavor — 
poorly measured, managed and rewarded. This paper considers the implementation of an 
academic advising program which parallels the 360-Degree feedback approach drawn from the 
Human Resources Management discipline. The details of the program are outlined and 
preliminary results of the program, which literally transformed academic advising at our 
institution, are discussed. 


THE CURRENT STATE OF ACADEMIC ADVISING 


yy 'n a broad sense, many of the primary research issues regarding academic advising at the University level 
have not changed much over the past 30 years. Then as now, advising researchers considered the 

f_ S difficulties of advising a heterogeneous student body, the lack of an explicit reward structure for advising, 

and the use of technology to improve advising (e.g., Lacher 1978; Teague and Grites 1980; and Sewart 1980 
respectively). 


However, recent trends accentuate the importance of investigating these issues. Increasing student 
employment loads, transfer rates, and emphases on universal student access tend to compound the complexities of 
advising. Meanwhile, increased research expectations, even in institutions whose mission has traditionally been 
teaching-focused, have resulted in a substantial reduction in time devoted to advising compared to the average time 
of faculty members measured 20 years ago. (Milem, Berger and Day 2000) 

The changing nature of high education has made academic advising more difficult due to changing 
missions, students, faculty, and delivery. However, unease over the adequacy of academic advising is nothing new. 
For instance, Russel and Thomas (1979) expressed serious reservations over the efficacy of faculty advising for 
first-year students. 

What is new is the confluence of trends making academic advising more challenging today that at any other 
modern time. University students have to focus more on outside employment—fifty percent of undergrads have 
federal loans compared to about 30 percent from 10 years ago (Livingston 2006). Students are taking longer to earn 
that “4-year” degree—nationally more than 40 percent of undergrads who do earn a degree are taking 5 or 6 years to 
do so( Knapp, Kelly-Reid and Whitmore 2006). Students are less academically ready for college—between 30 and 
60 percent of university students need academic remediation on entering the university (Conley 2006). These 
national trends among other complexities due to specific missions of individual institutions are exponentially 
increasing the complexity of advising. 


Meanwhile, determining the overall objective of academic advising has become more complex as well. 
Academic advising provides an important role in efforts to increase student access, retention and graduation rates, as 
well as to decrease both the time students take to earn a degree and the overall cost of providing a college education. 
Moreover, in a time of changing institutional missions it is difficult for many academic advisors to know the 
organizational mission much less align to their advising behaviors with that mission. 
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Our institution is a large, publicly funded university located in the southern United States. We believe that 
the state of advising at our school two years ago is typical of that at many other universities today. 

Despite its stated importance, the quality of faculty advising was not measured or reported at the 
University or College level. Much of this omission can be attributed to the higher weight usually given other 
faculty activities. In many faculty and administration minds, student advising received little weight 
compared to that given research, teaching, grant writing and university service. And, in this environment, it 
is unlikely that advising would play a significant role in performance appraisals. 

The primary recognition given for advising was an annual, university-level, advising award. The 
award itself was given at an honors convocation sparsely attended by faculty members. Our experience with 
this award underscored the wide variety of approaches to advising on the part of our faculty. 

We realized that we had the following challenges: 

1) The need to implement an organizational-wide change; 

2) A focus on advisor development rather than appraisal; 

3) the varying needs of students and styles of faculty required a holistic approach as there was no single 
“correct” way to advise; and, 

4) The need to get the “buy-in” of our faculty. 

A FORMAL MANAGEMENT APPROACH 

The 360-Degree Feedback approach provides a consistent management philosophy meeting the criterion 
outlined previously. The 360-degree feedback appraisal process describes a human resource methodology that is 
frequently used for both employee appraisal and employee development. Used in employee performance appraisals, 
the 360-degree feedback methodology is differentiated from traditional, top-down appraisal methods in which the 
supervisor responsible for the appraisal provides the majority of the data. Instead it seeks to use information gleaned 
from other sources to provide a fuller picture of employees’ performances. Similarly, when this technique used in 
employee development it augments employees’ perceptions of training needs with those of the people with whom 
they interact. 

The 360-degree feedback approach has two fundamental tenets: 1) the most effective approach to changing 
behavior is through feedback; and, 2) the various stake holders interacting with an individual provide the richest 
source of information for this feedback. (See Figure 1). 

Research has demonstrated measurable benefits to the technique; for instance, Brett (2006) measured a 
positive correlation between the use or 360 degree feedback and the increases in job satisfaction. Another advantage 
to the approach is that the use of many inputs tends to reduce rater bias (Taversky and Kahnneman 1974). 
Eischenger (2004) shows that rater accuracy tends to decrease over time, with those who have known the employee 
for more than five years to be least accurate in providing feedback. 

The 360-Degree approach is particularly well-suited to help bring about change to a long standing problem. 
As Jack Welch puts it - “Its [360-Degree Feedback] main value is to "out" the unspoken.” 

Criticisms of 360-Degree feedback generally are focused on its use in appraisal rather than development 
settings. These drawbacks include the possibility of too much negative feedback as well as employees “gaming” the 
system. That is, since employees rate each other they may enter into agreements that result in mutual back- 
scratching. 
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Figure 1 - Standard Performance Feedback for 360-degree Appraisal 


IMPLEMENTING A 360-DEGREE APPROACH TO ADVISING APPRAISAL: 

More than any other factor, executive sponsorship determines the success or failure of changes introduced 
into organizations (Hickens 1998). The reordering of priorities to emphasize advising is a monumental cultural 
change in an academic institution. Several years ago advising was identified as an important opportunity for 
improvement of our institution and we were fortunate enough to have our university president participate at the 
Board of Reagents level to lead our change. 

Our president also provided more tangible leadership, committing $50,000 per year for advising awards. 
Faculty members with more than 3 years of experiences were eligible for 1 of 45 one-thousand dollar awards. 
Newer faculty members were eligible for 20 five-hundred dollar awards. Additionally he agreed to sponsor and 
participate in an annual awards ceremony for the winning advisors. In addition to inviting the winning advisors and 
their family members, deans and department heads were also encouraged to attend. Now armed with the full 
attention of administration and faculty it became possible for us to develop our full advising plan using the new 
advising awards as our catalyst for change. 

As shown in Figure 2, our advising program uses a 360-degree appraisal system. The four main inputs into 
this process are provided by Department Heads and Deans (Supervisors), Students (subordinates and customers), the 
Advising Committee (peers), and academic administration. 

Students provide an important source of information, having an opportunity to rate their advisor when they 
register online. The rate their advisors on a 10-point Likert scale on 9 items as shown in Table 1. 


_ Table 1-Student Rating Items _ 

Knowledge of major requirements: 

Knowledge of course offerings and prerequisites: 
Punctuality for scheduled appointments: 

Awareness of my academic progress: 

Amount of time spent with me is sufficient: 

Listening skills and ease of conversation: 

Suggestion of alternatives with academic difficulties: 
Suggestion of resources for assistance: 

Help in preparing you for life after graduation: _ 
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The Advising Office shepherded the new advising plan. To foster increases in advising quality, the 
advising office published a list of screening criteria to be used in determining the winners of the awards. These 
criteria included attending a minimum number of advising workshops, advising a minimum number of students, and 
giving additional consideration to advisors who participated in special advising sessions. 

Deans and department heads provided rankings of all advising faculty members. 

The advising committee provided the final determination of award winners based on the rankings provided 
by deans and department heads, the ratings given by students and statements from applicants detailing their 
academic advising philosophies, methods and innovations. Three faculty members, three students, two academic 
deans, two professional advisors and a representative of the Advising Office make up the advising committee. 



Figure 2 - Advising 360-degree Feedback 


EMPIRICAL RESULTS 

The empirical results discussed in this paper focus on the student perceptions of advising from responses to 
an advising survey offered during registration. 

Confirmatory Factor Analysis: 

To verify that the nine items in the advising survey loaded on a single construct, a confirmatory factor 
analysis was conducted, resulting in a single factor accounting for 74.7% of the variance (Table 2). These results 
are consistent with a single underlying construct being largely responsible for student perceptions of advising. 
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Table 2 - Factor Analysis of Instrument Items 


Component 

Initial Eigenvalues 

Extraction Sums of Squared Loadings | 

Total 

% of Variance 

Cumulative % 

Total 

% of Variance 

Cumulative % 

1 

6.723 

74.704 

74.704 

6.723 

74.704 

74.704 

2 

.578 

6.420 

81.124 




3 

.461 

5.120 

86.244 

4 

.347 

3.856 

90.100 

5 

.261 

2.897 

92.997 

6 

.193 

2.140 

95.137 

7 

.162 

1.796 

96.933 

8 

.155 

1.726 

98.658 

9 

.121 

1.342 

100.000 


Extraction Method: Principal Component Analysis. 


DIFFERENCES BY CLASS 

A principal question addressed in the research is whether students at different levels perceive the quality of 
advising differently. Specifically, it was hypothesized that differences, if they existed, were likely to occur either as 
student progressed through their degrees. Specifically, it was thought that seniors and graduate students were likely 
to rate their advisors more favorable as they were likely to have formed stronger relationships with faculty and 
advisor than other undergraduate students. 

To consider this question, students were grouped by class and then their means on overall perceptions of 
the advising process were compared. A surrogate variable for overall perceptions was constructed by averaging 
respondents’ responses on the 9 items previously shown to load on a single factor. The simple means are reported in 
Table 3. 


Table 3 - Overall Perceptions of Advising by Class 


Class Standing 

Mean 

N 

Std. Deviation 

Freshman 

8.8661 

2776 

1.39563 

Sophomore 

8.8469 

2211 

1.50357 

Junior 

8.9044 

2061 

1.48525 

Senior 

9.0244 

2009 

1.52921 

Graduate 

9.4475 

539 

1.01019 

Total 

8.9357 

9596 

1.45741 


The reported means suggest significant differences may exist between classes. Thus, a standard one-way 
Analysis of Variance was conducted with class standing used for its factors followed by Tukeys HSD, a post hoc 
test allowing computation of significance between groups. The results of this test are summarized in table 4. 


Table 4 - Significance of Tukey Mean Comparisons 


Class 

Freshman 

Sophomore 

Junior 

Senior 

Graduate 

Freshman 

- 

.990 

.894 

.002 

.000 

Sophomore 


- 

.696 

.001 

.000 

Junior 



- 

.063 

.000 

Senior 




- 

.000 

Graduate 





- 


These results show the significance of differences in the mean satisfaction reported in Table 2. Thus, 
graduate student means are significantly higher than those of all other groups and seniors have significantly higher 
means than sophomores and juniors. These results agree with the hypothesis that students typically will rate 
advising higher the further they have progressed in college. 
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Analysis was also performed on ratings from transfer students compared to those of students who originally 
enrolled at the University. Table 5 shows the outcome — Although transfer students rated their advisors lower, the 
difference was insignificant fp = .340). However, Levene’s test showed a significant difference between the 
variances of the two groups (p=.011). 


Table 5 - Overall Advising Perceptions by Transfer Status 


Transfer Status 

Mean 

N 

Std. Deviation 

Transfer 

8.9005 

1163 

1.55772 

Non-Transfer 

8.9440 

8364 

1.44112 

Total 

8.9387 

9527 

1.45584 


CHANGES IN PERCEPTIONS OF ADVISING 

Before the changes to the student instrument, student responses were collected using a 4 point Likert scale 
(Strongly Agree, Agree, Disagree, and Strongly Disagree). Our revised instrument uses a 10 point scale. Any 
comparisons of means between these instruments would be problematic—even scaling would not solve the problem 
of taking essentially ordinal data and converting it to interval. Thus, rather than comparing means, we decided to 
use a Pearson’s Chi Square test by recoding the overall perceptions on the surveys into two categories—one above 
each surveys’ midpoint and one below each survey’s midpoint. The result was to combine Agree and Strongly 
Agree as well as Disagree and Strongly Disagree on the data obtained from a prior year and compare that to the 
groups determined by 1-5 and 6-10 on the new survey. While not as statistically strong as a means comparison, this 
chi square test is much more statistically valid. 

The results of this Chi Square test are reported in Table 6. This table demonstrates a huge increase in 
student satisfaction - for instance about 8% of the students selected either disagree or strongly disagree for their 
level of satisfaction; however, less that 3% of the students taking the new survey categorize their overall satisfaction 
as negative. The resulting Chi Square statistic was highly significant (p=.000). 


Table 6 - Comparison of Student Satisfaction (Old and New Surveys) 


Observed/Expected Values 

New Instrument 

Old Instrument 

Sum 

Not Satisfied 

271/451 

422/242 

693 

Satisfied 

9140/8959 

4614/4794 

13754 

Sum 

9411 

5036 

14447 


CONCLUSION, LESSONS LEARNED AND FUTURE RESEARCH 

One of the main advantages of a 360-degree feedback is the mutual investment in the outcome of the 
appraisal by all stakeholders. The contributions of so many stakeholders to a common enterprise create a level of 
energy that almost takes on a life of its own. 

As shown in Table 6, there has been a tremendous increase in the student’s perception of advising since 
implementing the new program. There has also been a corresponding increase in faculty efforts. Faculty 
attendance in advising workshops has markedly increased, from 210 in 2004 to 354 in 2005 and 641 in 2006. By all 
measures, this program has been incredibly successful. 

A major point of further research is finding out if there is any way to reach a small group of faculty that 
score far below the rest of their peers on the various components to this appraisal process. Similarly, there is a 
relatively small group of students who seem very dissatisfied with their advisors, is there a way to characterize these 
students that would make it possible to increase their satisfaction with the advising process? Another important 
trend is students concern with employability, this may cause us to rethink the end point of the advising process as 
well as the services provided seniors and graduate students as part of advising. 
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Of course, better advising should be thought of as a means enabling the ultimate end of improving student 
performance in school—leading to better grades, increased satisfaction, and increased retention. The current study 
stops well short of assessing these macro-level organizational gains. But, it does demonstrate that a well constructed 
advising program that follows the mantra of 360-degree feedback by soliciting the participation of all relevant 
stakeholders and using that feedback to construct a rich appraisal system can lead to substantial increases in advising 
quality. 
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